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Abstract 

This paper will explore some of the practical options that are available to 
teachers as we move towards Assessment 2.0. Assessment 2.0 describes an 
environment in which the teacher sets tasks that allow students to use more 
dynamic, immersive and interactive environments for exploring and 
creating responses to sophisticated assessment tasks. Assessments will 
become more like sophisticated games incorporating role-playing and 
scenarios; they will replicate many of the complexities of the real world, 
allowing students to explore and describe the consequences associated with 
their responses. This paper looks at how teachers can begin this journey 
through the use of simple interactive tools incorporated into familiar 
question types available in common learning management systems. 
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Introduction 

Assessment tasks in higher education must satisfy a diverse range of needs for a variety of 
stakeholders, including students, staff, institutions and future employers. Designing an assessment 
task is easy, however designing a meaningful assessment task that adds value for each of the 
stakeholders above is a time-consuming process and involves “planning, discussion, consensus 
building, reflection, measuring, analyzing, and improving based on the data and artefacts gathered 
about a learning objective” (Ramakishnan &Ramadoss, 2009). For teachers to be able to construct 
a variety of assessment tasks that will meet the needs of various stakeholders, they will likely have 
to devote significant time that will take them away from other activities, such as discipline 
research. So the temptation for time-poor teachers is to design assessment tasks that can be 
prepared quickly and that require student responses that can be marked efficiently. Meaningful 
assessment tasks that had an intrinsic worth beyond the immediate requirement for a mark or grade 
and which engaged students in an authentic activity would provide much deeper insights, for both 
the student and the teacher, about the capabilities that the student had developed. 

E-assessments have traditionally been used for tasks that focus on testing the acquisition of 
declarative knowledge, or knowing “what” (Bull & McKenna, 200; Northcote, 2003); such tasks 
have required students to select a predetermined response based on factual recall, for example the 
familiar multiple-choice (MCQ) and short answer question type. Such questions have been popular 
because they are quick to write and are easily constructed in common learning management 
systems used in higher education institutions; teachers do not need significant training to be able to 
write and deliver such assessments. Although MCQs can be quick to write, valid and reliable 
MCQs require considerable expertise to construct (Considine, Botti, & Thomas, 2005; Johnstone, 
2003). More recently, innovative e-assessment formats have been developed, including drag-and- 
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drop, hotspot, matrix or extended matching questions, voice responses, as well as the use of 
certainty -based marking to capture the students’ confidence in their level of understanding 
(http://www.ucl.ac.uk/lapt). Although these more innovative question types are becoming more 
commonly available in learning management systems, the predominant question type used is still 
the MCQ. 

MCQs are typical selected response questions; students are provided with options from which they 
make a choice. Such questions are typically associated with convergent responses, that is, every 
student is expected to make the same choice - there is no ambiguity about the expected response. 
Another way of referring to these questions is to define them as testing non -contestable 
knowledge, or the currently held truths in a discipline. In higher education there is clearly a body 
of knowledge associated with each discipline, and students are expected to become familiar with 
this knowledge over the course of their studies. However, higher education is also concerned with 
contestable knowledge or contestable ways of thinking and even what is non-contestable today is 
often contested over time as our understanding of a particular discipline becomes more refined. 
Why discuss this in a paper about assessment? This paper is concerned with sophisticated 
assessment tasks, in particular those undertaken using a computer and the web. In order to test 
higher order capabilities we need to design sophisticated assessment tasks, but the workload in 
designing such tasks is considerable. So this paper is proposing a design strategy for interactive e- 
assessments that uses many of the question types that teachers are familiar with and which are 
constructed using any of the common learning management systems. The difference is to include 
an additional digital tool within the question to provide teachers with the opportunity to ask more 
sophisticated questions and students with the opportunity demonstrate higher order capability 
development. For the purposes of this paper we have not distinguished between the different types 
of assessment, namely diagnostic, formative and summative. From the perspective of designing 
sophisticated e-assessment tasks it makes no difference whether the task is designed for current or 
future learning, or to provide a mark or grade for progression or completion of a program. 

General characteristics of sophisticated e-assessment tasks 

So what is a sophisticated e-assessment task? Boyle and Hutchison (2009) have described 
sophisticated e-assessment tasks as having the following characteristics: 

• should contain media-rich stimulus material (whether graphical, sound, video or 
animation) 

• the test taker should be required to interact with the stimulus material in a variety of ways 

• tend to be expensive and slow to develop, and not easily written by a non-specialist 
teacher 

Boyle and Hutchinson also pointed out that many sophisticated e-assessment tasks tend to generate 
a relatively large quantity of data about what the student did during the task, often making 
interpretation and the assigning of a mark or grade difficult for teachers. Boyle and Hutchinson 
provided two examples of sophisticated e-assessment tasks using purposely built software 
systems; the Tripartite Interactive Assessment Development system (TRIADs), work carried out at 
the University of Derby (http://www.i41earn.co.uk) and World Class Tests taken by gifted and 
talented teenagers in the United Kingdom (World Class Arena, 2004). Neither of these systems is 
incorporated into common learning management system (LMS), and teachers would be confronted 
with a significant learning curve themselves if they wished to construct e -assessment tasks using 
these packages. They do offer good examples of the type of sophisticated tasks that could be 
devised for students in the online environment. This provides actual examples of what could be 
done, but no real practical way for discipline teachers to use these systems without first investing 
considerable amounts of time becoming proficient in the use of the software. 

Currently, sophisticated e-assessment tasks are expensive and slow to construct, and frequently out 
of reach for discipline academics in universities. The technology is limiting a more widespread 
adoption of sophisticated e-assessment, so what is required is a design process that makes use of 
the current skills of teachers, or at least skills that that they can readily develop, and utilizes 
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existing LMSs and tools that are readily available, versatile, easy to use and reuse, and that 
teachers do not have to build. 

Assessment 2.0 

In the constructivist approach to learning, students are expected to make decisions and reflect on 
the consequences of those decisions (Rust, O’Donovan, & Price, 2005). A constructivist learning 
environment provides students with access to information and authentic learning tools. These same 
tools and information sources should be available for students to use when they undertake 
assessment tasks in order for students to demonstrate the development of higher order capabilities. 
One of the ways we can render e-assessment tasks more sophisticated is to provide students with 
tools that they can use to construct non-text responses, analyse data, interact with digital objects, 
or interrogate objects within an assessment task. Assessment 2.0 is an emerging term (Elliot, 2008) 
used to describe tasks that are aligned with the characteristics of the Web 2.0 environment; these 
are tool-assisted tasks that provide students with opportunities for solving authentic and 
personalised problems; they facilitate deep approaches to learning as the tasks require an 
understanding of how to use a tool to assist with the construction of the response. 

Allowing students to manipulate data, to examine the consequences of their responses and to make 
informed decisions about potential solutions are all consistent with the higher education ideals of 
assessing advanced skill development in students, as described by the higher levels of the SOLO 
taxonomy (Biggs, &Tang, 2007) or the Bloom’s taxonomy (Thomas, Ashton, Austin, Beevers, 
Edwards, & Milligan, 2004). Table 1 provides a summary of descriptors for sophisticated e- 
assessment tasks, using terms that are common in current assessment rubrics. Simulations and 
sophisticated digital tools allow students to construct multistructural and relational responses to 
questions. Sophisticated e-assessment tasks are not meant to replicate paper-based assessments; 
they are designed to make use of the characteristics of the new digital educational environment, 
namely interactivity. 


Table 1. Descriptors for sophisticated e-assessment tasks using digital tools (using modified 
Bloom’s taxonomy, Thomas et ah, 2004) 


Assessing 

Understanding 

Exemplify 

Predict 

students generate response data by using the tool 
students use the tool to predict what will happen 


Compare 

students use the tool to compare data for two scenarios 


Explain 

students use the tool to match selected response options in 
extended response questions type 

Assessing 

Application 

Complete 

students use the tool to process a dataset and generate a 
result as a response to the question 


Implement 

students use the tool to make a decision or draw a 
conclusion to an unknown situation 

Assessing 

Analysis 

Differentiate 

students are presented with a case study and use the tool 
to test a number of potential explanations 


Determine 

coherence 

students find relationships between information they have 
using the tool 

Assessing 

Evaluation 

Rationalise 

students use a dataset within a case study to justify trends 
or conclusions 


Critique 

students use the tool to look for inconsistencies in a 
dataset, scenario, or case study 

Assessing 

Creativity 

Exploring 

solutions 

students use the tool to assist them construct a hypothesis 


Planning and 
refining 

students use the tool to test a hypothesis and refine it 
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By providing students with tools that they must use to generate responses to questions, we are able 
to document the development of competencies and attributes that support current and future 
learning; students are actually developing meta-cognitive strategies for learning. One of the issues 
with many current assessment designs is that they foster a dependency in students on the teacher; 
this dependency is related to students believing they cannot make judgements about their own 
learning or performance level without input from the teacher. Clearly teachers have a pivotal role 
in defining standards and making judgements about assessment responses, but students should also 
be provided with opportunities to develop skills in judging their own learning and performance 
levels (Bond, & Palchikov, 2007). The ability of students to assess their own learning is critical to 
developing effective approaches to future learning. The difficult part for many teachers is 
designing appropriate learning activities that scaffold the development of these meta-cognitive self 
analysis skills and then to construct assessment tasks that are aligned with these learning activities. 
Although authors such as Shute, Ventura, Bauer, and Zapata-Rivera (2009) have proposed elegant 
models for measuring skills such as self-regulation or self-explanation, these tools have not been 
translated to a format suitable for use by discipline academics, nor have they been incorporated 
into common LMSs. Using sophisticated e-assessment tasks created through a common LMS 
should provide a more accessible pathway for discipline academics to design tasks that facilitate 
the testing of higher order capabilities and also provide students with access to tools that enable 
them to make judgements about their own learning or performance level. 

So how do we provide teachers with a design strategy for sophisticated e -assessment tasks that 
makes use of existing LMSs and does not increase their workload significantly? If we separate the 
interactive tool that the student will use to assist them in generating their response from the actual 
question by incorporating a web link (URL) to the tool within the question, the question and any 
feedback given by the teacher can be constructed in any quiz tool in any LMS (Figure 1). This 
design principle also requires only one copy of the interactive tool to exist on an institutional 
server and so it can be used many times for different purposes or with different groups of students. 


student 
responds to 
question in 
LMS 



Question 
constructed 
in LMS 


I 


I 


Student 
links tool 
and task 



URL link in 
question to 
digital tool 


Figure 1 . Design model for sophisticated e-assessment task 
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Discipline examples of sophisticated, interactive e-assessments 

Not all digital tools will be suited to e-assessment tasks, nor will all discipline areas necessarily be 
able to find freely available resources. Nevertheless, teachers will find that there are a significant 
number of suitable tools for many of the fundamental science, engineering and mathematics areas. 
An early example of the use of interactive Java applets in learning was presented by Healy, 

Berger, Romero, Aberson, and Saw. (2002). A Java applet is a small computer program that is 
written using the Java programming language and packaged so that it will execute through a web 
browser. It is very convenient for teachers and students when the digital tool to be used in a 
sophisticated e-assessment task is available in the form of a Java applet or a Flash file as no special 
software, other than a web browser, is required to use the tool. 


Many educational Java applets are freely available through the web and teachers can readily find 
and use these educational tools by either linking to the tool through an embedded URL to an 
external web site, or requesting a copy of the appropriate Java applet (*.jar or *. class files) for use 
on an institutional server. Teachers will not need to learn how to write their own Java applets, 
there are many suitable tools already available. The main technical task for the teacher is to embed 
the appropriate URL (that calls the Java applet) into an e-assessment item in their LMS. Figure 2 
illustrates a typical example of a question constructed in a common LMS, incorporating a URL 
link to a Java applet; in this case the applet is a simulation of diffraction patterns resulting from 
light of different frequencies being displayed through a slit of various widths and at different 
angles. The Java applet is available from http://www.walter-fendt.de/phl4e or it can be 
downloaded (with permission from the applet creator) to an institutional server. 


^3 9 

Marks: 

--/I 


Use the diffraction appiet beiow to match the following statements. 
Diffraction of iight by a singie siit 


For a slit with a 4000nm width and a waveiength 
of 600nm, the intensity of the diffraction lines 

For a slit where the waveiength of iight is 600nm 
the minimum siit width to generate 4 diffraction 
lines is 

For a slit with a 600nm width, the minimum 
waveiength of light to generate a diffraction 
pattern is 

For a slit with a lOOOnm width, increasing the 
waveiength of iight 


Choose... 


Choose... 

-419nm 


increases the number of diffration line 


1‘176nm 

decreases as the angle increases 


Choose... 




Submit 






r 


1 


i 


1 

90 ° 

60 ° 

30 ° 

0 ° 

30 ° 

60 ° 




Figure 2. Example of Java applet simulation embedded in an e-assessment question 
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From the point of view of the student using the Java applet in an e-assessment, it makes no 
difference whether the tool is located on a local server within the institution or it is located on a 
remote server across the world. The only issue for teachers to consider is whether access to the 
digital tool is required for a high stakes summative task; in this case it is likely to be more reliable 
to locate a copy of the tool on the institutional server as access to the tool can be controlled more 
readily at the local level. For low stakes summative or formative assessments, ensuring access to 
the tool at any time will be less critical. 

Table 2 provides some readily available sources for educational applets. 

Table 2: Examples of educational digital tools suitable for sophisticated e-assessment tasks 


Discipline 

URL 

Physics 

http://webphysics.davidson.edu/Applets/Applets.html, 

http://www.walter-fendt.de/ph14e/ 

http://www.phy.ntnu.edu.tw/ntnuJava 

Chemistry 

http://www.chemcollective.org/applets/vlab.php 

http://www.chemcollective.org/applets/pertable.php 

http://www.molinspiration.com/jme/ 

http://jmol.sourceforge.net/ 

Biological sciences 

http://wishart.biology.ualberta.ca/cgview/ 

http://employees.csbsju.edu/hjakubowski/Jmol/ERTamox/ 

3ERTnew.htm 

http://www.physionet.org/physiotools/ecgsyn/Java/ecgsyn- 

Java.html 

http://relax.organ.su.se:8123/eurocarb/gwb/builder.action 

Mathematics 

http://www.analyzemath.com/ 

http://www.geogebra.org/cms/ 

Engineering 

http://www.jhu.edu/virtlab/bridge/bridge.htm 

http://www.falstad.com/mathphysics.html 

http://www.engapplets.vt.edu/ 


Teachers should be aware that students will need to be given adequate time to become familiar 
with the Java applets before they can be used in an e -assessment task, as the tool is a crucial 
component of the assessment task. The more sophisticated the Java applet and the more options 
that are available for students within the applet, the more time teachers will need to allocate to 
learning activities that allow students to practice using the tool. 

The early study by Healy et al. (2002) highlighted that the use of a digital tool per se, does not 
automatically improve student learning. It is critical that appropriate design strategies are used to 
ensure that any digital tool is pedagogically effective. Buzzetto-More and Alade (2006) collated 
examples of good practice in e-assessment and highlighted the advantages of the use of 
simulations as authentic activities. Simulations or digital tools that allow the analysis of datasets 
require students to use higher order thinking as well as the application of knowledge and skills in a 
potentially authentic environment. However, the use of a simulation or digital tool that is 
associated with a simple MCQ will rarely give any insight into the thought processes students used 
to determine their responses. One option that teachers can use is to construct nested selected 
response questions, such as those illustrated in Figure 3, where the student is provided with a 
series of interrelated questions, each requiring the use of the digital tool, and each aligned with a 
key concept, in this case the structure -activity relationship between functional groups and 
solubility. This example uses the traditional format of a computer-marked question format, so uses 
the selected response format. 
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The example in Figure 3 also highlights the potential of incorporating digital tools and simulations 
into e-assessments to radically change the type of question we can set for students. The traditional 
selected response format (MCQ) is associated with convergent responses, that is, all students are 
expected to provide the same (correct) response. This is true for the questions illustrated in Figure 
3. However, as we have provided students with a tool that allows them to explore the relationship 
between structure and activity themselves, we could reframe this question to require a divergent 
response; different students could provide different, equally valid, responses. If we asked the 
students to design a molecule that had specific properties, for example, a TPS A value between 15 
and 17, then students would need to be familiar with the relationship between structure and the 
property being investigated, but they are able to demonstrate that understanding by designing their 
own molecule. In this case, the students’ responses would be assessed by a teacher, rather than the 
computer. Students can also test their own understanding by changing the structure of the molecule 
and examining the effect on physical or biological properties. This activity would facilitate the 
development of self-review skills in students; by providing students with tools to test their own 
levels of understanding we would be reducing their dependency on teachers, and having to wait to 
receive feedback about whether they have an adequate understanding of the key concepts. 


Marks: 

-/I 

Draw the stuctures of the two molecules labelled as A and B using the JME applet available on 
the following web page: 


http : //WWW . mo 1 1 nsp i rat I on . com/cg i -b I n/propert I es 



/-X 




A 

B 



Use the applet to predict the logP values for each molecule and the predicted bioactivity. Use 
this information to match the following statements. 


The presence of a sulphonimide group 

1 Choose... 



results in a 




Molecuie B is a 
Molecuie A is a 

lower logP value 

better GPCA ligand compared to the other molecule 
higher logP value 

worse GPCA ligand compared to the other molecule 



fnioFrf^,|: iration 

SMILES FC(F)(F)c2ccc(c1 ccc[nH]1 )cc2 

Molinspir ation property engine v2009.01 




miLogP 

3.421 




TP3A 

15.791 




natoms 

15 


1 

/ V 

MW 

211.186 




nON 

1 


F 

AqHP 

nOHNH 

1 



nviolations 

0 


1 


nrotb 

2 



rl 

volume 

171.735 




Get data as 

text (for copy / paste). 



mal 






Get 3D geometry BETA 


Figure 3. Example of digital tool embedded in an e-assessment question 

In order to reduce the workload on teachers when they are constructing sophisticated e -assessment 
tasks, institutional learning designers or other support staff, could develop templates that are 
readily incorporated into the institutional LMS. An example is illustrated in Figure 4. A series of 
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such templates could be made available for different question formats using a particular digital 
tool; this digital tool would likely be one that resides on the institutional server and used for both 
learning and assessment activities. This approach of developing templates for teachers has a 
number of benefits; it encourages a team approach to curriculum design, it reduces the need for 
teachers to learn technical skills before being able to generate sophisticated e-assessment tasks for 
their students, it also maintains the role of the teacher as the person responsible for writing the 
actual question and providing the appropriate feedback. Teachers will feel they are still in control 
of the learning environment, even if they have only basic technical skills. 


^3 ^ 

Marks' JalView multiple alignment editor to match the following statements. 

Ask a relevant question. 


Insert a relevant statement 
Insert a relevant statement 
Insert a relevant statement 
Insert a relevant statement 
I Submit I 


[ChooseTI^ 


Choose... 


Answer 3 
Answer 2 
Answer A 
Answer 1 


File Edit Select View Format 
10 

asyIvIl I tp^o 
ASY vIl I TP do 
ASY vIl I TPEO 
A5.Y vIlITPDO 


Colour Calculate 
20 


ATY 

A-TY 

ASY 

ATY 

AT 


AAjy VTLVTPEO 
AA^VJL L VT P 

F I TPEO 
F I TPEO 


at*Jnv 

A-T'i^V 

AfYlV 


LVTPDO 
L I TPEO 
L I TPEO 
L I TPEO 
F I TPEO 
F I XPlO 


P I EF DCp|dTvY I L 
PIEFDCPDNVYIL 
P I EF ECPDDVY I L 
P I IF ECPDDVY I L 
KQE L ECPDDVY I L 
EFQCP.DDVY I L 

evecIdIddvyvl 
eqevecIeIedvyvl 

TQEF ECPDDVY I L 

PQ If dcpIddvy i l 

ELQVPDDVY I L 
E LQVP.DDVY I L 
DVYVL 
D.VYVL 



Help 

30 

dIqaeeaohd' 

DQAEEAOHD 

dqa.eIe^hd 

DQA.E^E^HD 
DAAEEAO I D 
DAAE^E^ I D 
DAAEEAO I D 
DAAEEAOLD 
DHAEEVO I D 
DHAEEVO I E 
HqAeId!^ I D 
DF AE ^Eil I D 
DAAEEAOLD 
.DAAEEAO I ^ 


40 

I L 

LPYSCRAO 
LPYSCRAO 
LPYSCRAO 
LPYSCRAO 
LPYSCRAO 
LPYSCRAO 
LPYSCRAO 
LPYSCRAO 
LPYSCRAO 
LPYSCRAO 
LPYSCRAO 
LPF SCRAO 
LPYSCRAO 

lp ys c.rIao 



F£R_CAPAA 

fER_OAPAN 

fER‘i_SOLl.C 

Q93XJ9_SOLTU 

FER1_f\SESCR 

fER1_SP/OL 

fER1_ARATH 

FER2_ARATH 

FER1_PEA 

Q?XA9Q_TRiPR 

FER1_MA/ZE 

O80429_MA/ZE 

Q93Z60_ARA TH 

FER3_RAPSA 

Lupas_21 
Lupas_14 
Lupas_28 
jnetpred 


JNETCONF 

1312779971689813551689748872320212S7887642135787S4457525514 

JNETSOL25 B-B - - BBBBB BBBB-B-B BBBBBBBBBBBBBBB - - B- - - 

JNETSOL5 bB B B 

JNETSOLO 

JNETHMM ■■■■■■► ■■■# ^ 

JNETALION ^ 4 4 4 

^ 



Figure 4. Example of digital tool embedded in an e-assessment question 


Java applets were used to illustrate the concept of a digital tool that could facilitate the 
construction of sophisticated e-assessment tasks. Other examples of relevant digital tools include 
the use of Excel spreadsheets with embedded macros (Lim, 2006), Elash simulations and 
QuickTime VR images. 3D images, where students could examine the relative spatial orientations 
of objects within the image, would allow more sophisticated questions to be set in the online 
environment. An example of the use of a QuickTime VR image incorporated into a selected 
response question in a typical LMS is illustrated in Eigure 5. Here the student is being asked to 
examine the volcano and identify particular geological features. Although this example is quite 
simple, the teacher could create a series of nested selected response questions that require the 
student to move around the 3D image of the volcano and identify objects that have specific spatial 
relationships to each other. This type of assessment task would not be possible with a static image 
of a site. Other examples of useful QuickTime VR images would be complex 3D images of 
anatomical or biological specimens, molecular structures, engineering sites, archaeological sites, 
museums, landscape features or architectural plans for buildings. 
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Examine the Quicktime VR image of the Ubehebe crater 



to answer the following question. 

http://www.exploratorium.edu/learning_studio/news/january98/ubehebe_crater.mov 
The layers of rock exposed in the crater show evidence for the presence of 


Choose one 
answer. 


O a. basalt 
O b. quartz 
O c. limestone 
O cl. conglomerate layers 


Figure 5. Example of QuickTime VR image embedded in an e-assessment question 


Conclusion 

This paper has proposed an e-assessment design model based on the incorporation of a URL link 
to a digital tool within questions developed using quiz tools in common learning management 
systems. The design allows teachers with minimal technical skills to construct sophisticated e- 
assessment tasks and to test higher order skills. Providing students with access to digital tools 
within assessment tasks also facilitates the development of self-analysis skills in students and 
reduces their dependence on teachers for affirmation of whether a response is appropriate or not. 
The modular nature of the design principle also allows the development of templates that teachers 
could use, further reducing the workload required to build online assessments. 
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